Vehicle driving assistance device and vehicle

ABSTRACT

Disclosed is a driver assistance apparatus for a vehicle, the apparatus including: a camera configured to acquire a front field-of-view image of the vehicle; an output unit; and a processor configured to detect at least one traffic sign based on the front field-of-view image of the vehicle, determine whether the traffic sign fits travel information of the vehicle, and, when the traffic sign fits the vehicle travel information, perform control to output the traffic sign through the output unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a National Stage application under 35 U.S.C. § 371of International Application No. PCT/KR2016/013737, filed on Nov. 26,2016, which claims the benefit of Korean Application No.10-2016-0060974, filed on May 18, 2016. The disclosures of the priorapplications are incorporated by reference in their entirety.

TECHNICAL FIELD

The present disclosure relates to a driver assistance apparatus providedin a vehicle, and the vehicle.

BACKGROUND ART

A vehicle is an apparatus that is motorized to move in a desireddirection by control of a user riding therein. A typical example of thevehicle is an automobile.

Vehicles have been increasingly equipped with various sensors andelectronic devices to provide user convenience. In particular, AdvancedDriver Assistance Systems (ADAS)s are under development. Furthermore,efforts are being made to develop autonomous vehicles.

Traffic Sign Recognition (TSR) is one of the ADASs. The TSR is a systemfor detecting a traffic sign based on an acquired image and informing adriver of the detected traffic sign.

However, a TSR according to an existing technology simply detects atraffic sign and outputs relevant information and is not capable ofoperating in an manner suitable for a traveling situation. Specifically,when two contradictory traffic signs are detected, both of the two signsare output, leading confusion to a driver.

In addition, as any traffic signs are recklessly detected and output,this may rather disturb a driver to focus on driving.

DISCLOSURE Technical Object

To solve the above problems, one object of the present invention is toprovide a driver assistance apparatus that outputs only a traffic signwhich fits a vehicle's travel information.

In addition, another object of the present invention is to provide avehicle including the above driver assistance apparatus.

Objects of the present invention should not be limited to theaforementioned objects and other unmentioned objects will be clearlyunderstood by those skilled in the art from the following description.

Technical Solution

To achieve the above objects, the present invention provides a driverassistance apparatus for a vehicle, including a camera configured toacquire a front field-of-view image of the vehicle; an output unit; anda processor configured to detect at least one traffic sign based on thefront field-of-view image of the vehicle, determine whether the trafficsign fits travel information of the vehicle, and, when the traffic signfits the travel information, perform control to output the traffic signthrough the output unit.

The details of other embodiments are included in the followingdescription and the accompanying drawings.

Advantageous Effects

According to embodiments of the present invention, there are one or moreadvantageous effects as follows.

First, only a traffic sign fitting a vehicle's travel information isoutput, thereby providing only information necessary for a driver andpreventing unnecessary information.

Second, only an information item fitting to a situation amongcontradictory information items is output, preventing confusion to adriver.

Third, when a plurality of traffic signs is detected, the plurality oftraffic signs are output according to a predetermined priority order,preventing an accident caused by distraction of a driver by the trafficsigns.

Fourth, a traffic sign fitting to a driver's intention is output,thereby providing information necessary for the driver and preventingunnecessary information.

Effects of the present invention should not be limited to theaforementioned effects and other unmentioned effects will be clearlyunderstood by those skilled in the art from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 shows the exterior of a vehicle according to an implementation ofthe present invention.

FIG. 2 is a block diagram illustrating a vehicle according to anembodiment of the present invention.

FIG. 3A is a perspective view of a camera for a vehicle according to anembodiment of the present invention. FIG. 3B is an exploded perspectiveview of a camera for a vehicle according to an embodiment of the presentinvention. FIG. 3C is a side view of a camera for a vehicle, which iscut along line A-B shown in FIG. 3A, according to an embodiment of thepresent invention.

FIG. 3D is a perspective view of a camera for a vehicle according to anembodiment of the present invention. FIG. 3E is an exploded perspectiveview of a camera for a vehicle according to an embodiment of the presentinvention. FIG. 3F is a side view of a camera for a vehicle, which iscut along line C-D shown in FIG. 3D, according to an embodiment of thepresent invention.

FIG. 4A is a block diagram illustrating a driver assistance apparatusaccording to an embodiment of the present invention.

FIG. 4B is a diagram illustrating internal configurations of a processorand a signal processing procedure by each configuration according to anembodiment of the present invention.

FIGS. 5A and 5B illustrate various examples of an internal block diagramof an image processor illustrated in FIG. 4B, and FIGS. 5C and 5D arediagrams illustrating operation of a processor illustrated in FIG. 5B.FIGS. 5E and 5F illustrate example operations of the driver assistanceapparatus illustrated in FIGS. 5A to 5D.

FIG. 6 is a flowchart illustrating operation of a driver assistanceapparatus according to an embodiment of the present invention.

FIG. 7 is a diagram illustrating an operation of detecting a trafficsign according to an embodiment of the present invention.

FIG. 8 is a diagram illustrating an operation of a driver assistanceapparatus in response to detection of a plurality of traffic signsaccording to an embodiment of the present invention.

FIG. 9 is a diagram illustrating an operation of outputting a trafficsign according to an embodiment of the present invention.

FIG. 10 is a diagram illustrating a driver assistance apparatus inresponse to acquisition of a plurality of items of travel informationaccording to an embodiment of the present invention.

FIG. 11 is a diagram illustrating an operation of outputting a trafficsign based on distance to a plurality of traffic signs according to anembodiment of the present invention.

FIG. 12 is a diagram illustrating an operation of outputting a trafficsign in response to detection of a plurality of traffic signs accordingto an embodiment of the present invention.

FIG. 13 is a diagram illustrating an operation of outputting an alarm bya driver assistance apparatus according to an embodiment of the presentinvention.

FIG. 14 is a diagram illustrating an operation of a driver assistanceapparatus in response to entrance to a rampway according to anembodiment of the present invention.

FIG. 15 is a diagram illustrating an operation of a driver assistanceapparatus in response to detection of a plurality of traffic lampsaccording to an embodiment of the present invention.

FIG. 16 is a diagram illustrating an operation of outputting a trafficsign fitting line information by a driver assistance apparatus accordingto an embodiment of the present invention.

FIG. 17 is a diagram illustrating an operation of a driver assistanceapparatus in a congested area according to an embodiment of the presentinvention.

FIG. 18 is a diagram illustrating an operation of a driver assistanceapparatus depending on a vehicle model according to an embodiment of thepresent invention.

BEST MODE

Hereinafter, the embodiments disclosed in the present specification willbe described in detail with reference to the accompanying drawings, andthe same or similar elements are denoted by the same reference numeralseven though they are depicted in different drawings and redundantdescriptions thereof will be omitted. In the following description, withrespect to constituent elements used in the following description, thesuffixes “module” and “unit” are used or combined with each other onlyin consideration of ease in the preparation of the specification, and donot have or serve as different meanings. In addition, in the followingdescription of the embodiments disclosed in the present specification, adetailed description of known functions and configurations incorporatedherein will be omitted when it may make the subject matter of theembodiments disclosed in the present specification rather unclear. Inaddition, the accompanying drawings are provided only for a betterunderstanding of the embodiments disclosed in the present specificationand are not intended to limit the technical ideas disclosed in thepresent specification, and it should be understood that the accompanyingdrawings include all modifications, equivalents and substitutionsincluded in the scope and sprit of the present invention.

It will be understood that although the terms “first,” “second,” etc.,may be used herein to describe various components, these componentsshould not be limited by these terms. These terms are only used todistinguish one component from another component.

It will be understood that when a component is referred to as being“connected to” or “coupled to” another component, it may be directlyconnected to or coupled to another component or intervening componentsmay be present. In contrast, when a component is referred to as being“directly connected to” or “directly coupled to” another component,there are no intervening components present.

As used herein, the singular form is intended to include the pluralforms as well, unless the context clearly indicates otherwise.

In the present application, it will be further understood that the terms“comprises”, includes,” etc. specify the presence of stated features,integers, steps, operations, elements, components, or combinationsthereof, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components, orcombinations thereof.

A vehicle as described in this specification may include an automobileand a motorcycle. Hereinafter, a description will be given based on anautomobile.

A vehicle as described in this specification may include all of aninternal combustion engine vehicle including an engine as a powersource, a hybrid vehicle including both an engine and an electric motoras a power source, and an electric vehicle including an electric motoras a power source.

In the following description, “the left side of the vehicle” refers tothe left side in the forward driving direction of the vehicle, and “theright side of the vehicle” refers to the right side in the forwarddriving direction of the vehicle.

FIG. 1 shows the exterior of a vehicle according to an implementation ofthe present invention.

Referring to FIG. 1, a vehicle 100 may include wheels rotated by a powersource, and a steering input device a for adjusting the travel directionof the vehicle 100.

According to an embodiment, the vehicle 100 may be an autonomousvehicle. If the vehicle 100 is an autonomous vehicle, the autonomousvehicle may be switched to an autonomous mode or a manual mode accordingto a user's input. When switched to the manual mode, the autonomousvehicle 100 may receive a driver's input for traveling through a drivemanipulation device 121 (see FIG. 2).

The vehicle 100 may include a driver assistance apparatus 400. Thedriver assistance apparatus 400 is an apparatus which assists a driverbased on information acquired by a variety of sensors. The driverassistance apparatus 400 may be referred to as an Advanced DriverAssistance System (ADAS).

In the following description, a camera 200 for a vehicle is mainlydescribed as a sensor used in the driver assistance apparatus 400, butaspects of the present invention are not limited thereto. According toan embodiment, a Radar, a LiDar, an ultrasonic sensor, and an infraredsensor as well as the camera 200 may be used as a sensor.

In addition, in the following description, a mono camera 200 a and astereo camera 200 b are mainly described as the camera 200 used in thedriver assistance apparatus 400, but aspects of the present inventionare not limited thereto. According to an embodiment, the camera 200 mayinclude a triple camera, an Around View Monitoring (AVM) camera, a350-degree camera, and an omnidirectional camera.

The drawing shows an example in which the camera 200 used in the driverassistance apparatus 400 is mounted onto a front windshield 10 so as tophotograph a front view of the vehicle, but the camera 200 mayphotograph any of the front view, the rear view, the right-side view,and the left-side view of the vehicle. Accordingly, the camera 200 maybe disposed at an appropriate position outside or inside the vehicle.

An overall length means the length between the front part and the rearpart of the vehicle 100, an overall width means the width of the vehicle100, and an overall height means the distance between the lower part ofthe wheel and the roof of the vehicle 100. In the following description,an overall length direction L may refer to a direction in which theoverall length of the vehicle 100 is measured, an overall widthdirection W may refer to a direction in which the width of the vehicle100 is measured, and an overall height direction H may refer to adirection in which the height of the vehicle 100 is measured.

FIG. 2 is a block diagram illustrating a vehicle according to anembodiment of the present invention.

Referring to FIG. 2, the vehicle 100 may include a communication unit110, an input unit 120, a sensing unit 125, a memory 130, an output unit140, a vehicle drive unit 150, a controller 170, an interface unit 180,a power supply unit 190 and the driver assistance apparatus 400.

The communication unit 110 may include a short-range communicationmodule 113, a location information module 114, an optical communicationmodule 115, and a V2X communication module 116.

The communication unit 110 may include one or more Radio Frequency (RF)circuits or devices to communicate with a different device.

The short-range communication module 113 is a module for short rangecommunication and capable of supporting short range communication usingat least one of Bluetooth™, RFID (Radio Frequency Identification),Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC(Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct andWireless USB (Wireless Universal Serial Bus).

The short-range communication module 113 may perform short-rangecommunication between the vehicle 100 and at least one external deviceby establishing wireless area networks. For example, the short-rangecommunication module 113 may wirelessly exchange data with a mobileterminal. The short-range communication module 113 may receive weatherinformation and traffic information (e.g., transport protocol expertsgroup (TPEG) information) from a mobile terminal. When a user enters thevehicle 100, a mobile terminal of the user and the vehicle 100 may bepaired automatically or according to execution of an application by theuser.

The location information module 114 is a module for acquiring a locationof the vehicle 100 and a typical example thereof is a GPS (GlobalPositioning System) module. For example, the vehicle can acquire thelocation thereof using signals sent from a GPS satellite using the GPSmodule.

Meanwhile, according to an embodiment, the location information module114 may be a component included in the sensing unit 125 instead of thecommunication unit 110.

The optical communication module 115 may include a light transmissionunit and a light receiving unit.

The light receiving unit converts a light signal into an electricalsignal so as to receive information. The light receiving unit mayinclude a photodiode (PD) for receiving light. The photodiode convertslight into an electrical signal. For example, the light receiving unitcan receive information on a preceding vehicle through light emittedfrom a light source included in the preceding vehicle.

The light transmission unit may include at least one light-emittingelement for converting an electrical signal into a light signal. Here,the light-emitting element is preferably an LED (Light Emitting Diode).The light transmission unit converts an electrical signal into a lightsignal and emits the light signal. For example, the light transmissionunit may emit a light signal through flickering of the light-emittingelement, which corresponds to a predetermined frequency. According to anembodiment, the light transmission unit may include a plurality oflight-emitting element arrays. According to an embodiment, the lighttransmission unit may be integrated with a lamp provided to the vehicle100. For example, the light transmission unit may be at least one of aheadlight, a taillight, a brake light, a turn signal lamp and asidelight. For example, the optical transmission module 115 may exchangedata with another vehicle through optical communication.

The V2X communication module 116 is a module for wireless communicationbetween the vehicle 100 and a server or other vehicles. The V2Xcommunication module 116 includes a module in which a vehicle-to-vehiclecommunication (V2V) or vehicle-to-infrastructure communication (V2I)protocol can be implemented. The vehicle 100 can perform wirelesscommunication with an external server or other vehicles through the V2Xcommunication module 116.

The input unit 120 may include a driving manipulation device 121, amicrophone 123 and a user input unit 124.

The driving manipulation device 121 receives user input for driving thevehicle 100. The driving manipulation unit 121 may include a steeringinput device, a shift input device, an acceleration input device and abrake input device.

The steering input device receives an input regarding a direction oftravel of the vehicle from a user. The steering input device ispreferably configured as a steering wheel such that steering inputaccording to rotation can be applied. According to an embodiment, thesteering input device may be configured in the form of a touchscreen, atouch pad, or a button.

The shift input device receives a parking (P) input, a drive (D) input,a neutral (N) input, and a reverse (R) input for the vehicle 100 fromthe user. The shift input means 121 b is preferably formed into a lever.According to an embodiment, the shift input means 121 b may beconfigured as a touch screen, a touchpad, or a button.

The acceleration input device receives an acceleration input for thevehicle 100 from the user. The brake input device receives adeceleration input for the vehicle 100 from the user. The accelerationinput device and the brake input device are preferably formed intopedals. According to an embodiment, the acceleration input means 121 cor the brake input device may be configured as a touch screen, atouchpad, or a button.

The microphone 123 may process an external audio signal into electricaldata. The processed data may be used in various manners according tofunctions executed in the vehicle 100. The microphone 123 may convert avoice command of the user into electrical data. The converted electricaldata may be transmitted to the controller 170.

Meanwhile, according to an embodiment, the camera 122 or the microphone123 may be included in the sensing unit 125 instead of the input unit120.

The user input unit 124 is configured to receive information from theuser. Upon input of information through the user input unit 124, thecontroller 170 may control an operation of the vehicle 100 incorrespondence with the input information. The user input unit 124 mayinclude a touch input means or a mechanical input means. According to anembodiment, the user input unit 124 may be disposed in an area of asteering wheel. In this case, the driver may manipulate the user inputunit 124 with his or her finger, while grabbing the steering wheel.

The sensing unit 160 may sense various situations of the vehicle 100 andsituations outside the vehicle. For this purpose, the sensing unit 160may include a collision sensor, a steering sensor, a speed sensor, aninclination sensor, a weight sensor, a heading sensor, a yaw sensor, agyro sensor, a position module, a vehicle forward/backward movementsensor, a battery sensor, a fuel sensor, a tire sensor, a steeringsensor, an in-vehicle temperature sensor, an in-vehicle humidity sensor,an ultrasonic sensor, an illumination sensor, an acceleration pedalposition sensor, a brake sensor, etc.

The sensing unit 160 may acquire sensing signals for vehicle collisioninformation, vehicle heading information, vehicle location information(GPS information), vehicle angle information, vehicle speed information,vehicle acceleration information, vehicle tilt information, vehicleforward/backward movement information, battery information, fuelinformation, tire information, vehicle lamp information, in-vehicletemperature information, in-vehicle humidity information, steeringinformation, external illumination, pressure applied to an accelerationpedal, pressure applied to a brake pedal, etc.

In addition, the sensing unit 125 may further include an accelerationpedal sensor, a pressure sensor, an engine speed sensor, an air flowsensor (AFS), an air temperature sensor (ATS), a water temperaturesensor (WTS), a throttle position sensor (TPS), a TDC sensor, a crankangle sensor (CAS), and the like.

The location information module 114 may be classified as a component ofthe sensing unit 125.

The sensing unit 125 may include an object sensor for sensing an objectaround the vehicle. The object sensor may include a camera module, aradar, a lidar and an ultrasonic sensor. In this case, the sensing unit125 can sense an object in front of the vehicle or an object behind thevehicle through the camera module, radar, lidar or ultrasonic sensor.

According to an embodiment, the object sensing unit may be classified asa component of the driver assistance apparatus 400.

The memory 130 is electrically connected with the controller 170. Thememory 130 may store basic data for a unit, control data for controllingan operation of the unit, and input and output data. The memory 130 maybe any of various storage devices in hardware, such as Read Only Memory(ROM), Random Access Memory (RAM), Erasable and Programmable ROM(EPROM), flash drive, hard drive, etc. The memory 130 may store variousdata for overall operations of the vehicle 100, such as programs forprocessing or controlling in the controller 170.

The output unit 140 is configured to output information processed by thecontroller 170 and may include a display device 141, a sound output unit142, and a haptic output unit 143.

The display device 141 may display various graphical objects. Forexample, the display device 141 can display vehicle related information.Here, the vehicle related information may include vehicle controlinformation for direct control of the vehicle or vehicle drivingassistance information for providing driving guidance to the vehicledriver. In addition, the vehicle related information may include vehiclestate information indicating the current state of the vehicle or vehicledriving information related to driving of the vehicle.

The display device 141 may include at least one of a liquid crystaldisplay (LCD), a thin film transistor-liquid crystal display (TFT LCD),an organic light emitting diode (OLED), a flexible display, a 3Ddisplay, or an e-ink display.

The display device 141 may form a layered structure with a touch sensoror be integrally formed integrated with the touch sensor so as toimplement a touch screen. The touchscreen may function as a user inputunit 724 providing an input interface between the vehicle 100 and theuser, and, at the same timey, provides an output interface between thevehicle 100 and the user. In this case, the display device 141 mayinclude a touch sensor for sensing a touch applied to the display device141 such that a control command is input to the display device 141through the touch. When a touch is applied to the display device 141,the touch sensor may sense the touch and the controller 170 can generatea control command corresponding to the touch on the basis of the sensedtouch. Input applied through touch may be text, figures or menu itemswhich can be instructed or designated in various modes.

The display device 141 may include a cluster to enable the driver todrive the vehicle and, simultaneously, to check vehicle stateinformation or vehicle driving information. The cluster may be providedon the dashboard. In this case, the driver may be able to checkinformation displayed on the cluster while looking forward.

Meanwhile, according to an embodiment, according to embodiment, thedisplay device 141 may be implemented as an HUD (Head Up Display). Whenthe display device 141 is implemented as an HUD, information can beoutput through a transparent display provided to the windshield of thevehicle. Alternatively, the display device 141 may include a projectionmodule so as to output information through an image projected onto thewindshield.

Meanwhile, according to an embodiment, the display device 141 mayinclude a transparent display. In this case, the transparent display maybe attached to the front windshield 10.

The transparent display may display a predetermined screen withpredetermined transparency. For transparency, the transparent displaymay include at least one of a transparent TFEL

(Thin Film Electroluminescent) display, a transparent OLED (OrganicLight-Emitting Diode) display, a transparent LCD (Liquid CrystalDisplay), a transmission type transparent display, or a transparent LED(Light Emitting Diode) display. The transparency of the transparentdisplay may be controlled.

According to an embodiment, the display device 141 may function as anavigation device.

The sound output unit 142 converts an electrical signal from thecontroller 17—into an audio signal and outputs the audio signal. To thisend, the sound output unit 142 may include a speaker or the like. Thesound output unit may be able to output sound corresponding to operationof the user input unit 724.

The haptic output unit 143 may generate a tactile output. For example,the haptic output unit 143 may vibrate the steering wheel, a safetybelt, or a seat to enable the user to recognize haptic output.

The vehicle drive unit 150 may control operations of various devices ofthe vehicle. The vehicle drive unit 150 may include a power sourcedriver 151, a steering driver 152, a brake driver 153, a lamp driver154, an air-conditioner driver 155, a window driver 156, an airbagdriver 157, a sunroof driver 158 and a suspension driver 159.

The power source driver 151 can perform electronic control of a powersource of the vehicle 100.

For example, when the power source is a fossil fuel based engine (notshown), the power source driver 151 can perform electronic control ofthe engine so as to control the output torque of the engine. When thepower source driver 151 is an engine, the speed of the vehicle can belimited by restricting an engine output torque under the control of thecontroller 170.

Alternatively, when an electric motor (not shown) is a power source, thepower source driver 151 may control the motor. Accordingly, revolutionsper minute (RPM), torque and the like of the motor can be controlled.

The steering driver 152 may electronically control a steering apparatusof the vehicle 100 so as to steer the vehicle 100.

The brake driver 153 may electronically control a brake apparatus (notshown) of the vehicle 100. For example, the brake driver 153 can reducethe speed of the vehicle 100 by controlling the operation of a brakeprovided to the wheels. As another example, the brake driver 153 mayadjust the direction of the vehicle 100 to the left or right bydifferently operating brakes respectively provided to the left and rightwheels.

The lamp driver 154 may turn on/turn off lamps provided inside andoutside the vehicle 100. In addition, the lamp driver 154 may controlilluminance, directions and the like of the lamps. For example, the lampdriver 154 may control a turn signal lamp, a brake lamp, and the like.

The air-conditioner driver 155 may electronically control an airconditioner (not shown) of the vehicle 100. For example, theair-conditioner driver 155 can control the air conditioner to supplycool air to the inside of the vehicle 100 when the interior temperatureof the vehicle is high.

The window driver 156 may electronically control a window apparatus ofthe vehicle 100. For example, the window driver 156 may control openingor closing of left and right windows provided to the side of thevehicle.

The airbag driver 157 may electronically control an airbag apparatusprovided inside the vehicle 100. For example, the airbag driver 157 maycontrol the airbag apparatus to be developed in a dangerous situation.

The sunroof driver 158 may electronically control a sunroof apparatus(not shown) of the vehicle 100. For example, the sunroof driver 158 maycontrol opening or closing of a sunroof.

The suspension driver 159 may electronically control a suspensionapparatus (not shown) of the vehicle 100. For example, the suspensiondriver 159 may reduce vibration of the vehicle 100 by controlling thesuspension apparatus when the surface of the road is rough.

According to embodiment, the vehicle driving unit 150 may include achassis driver. The chassis driver may include the steering driver 152,brake driver 153 and suspension driver 169.

The controller 170 may control operations of the respective units of thevehicle 100. The controller 170 may be called an ECU (Electronic ControlUnit).

The controller 170 may be implemented using at least one of ASICs(application specific integrated circuits), DSPs (digital signalprocessors), DSPDs (digital signal processing devices), PLDs(programmable logic devices), FPGAs (field programmable gate arrays),processors, controllers, microcontrollers, microprocessors, orelectrical units for performing other functions.

The interface 180 may serve as a passage between the vehicle 100 andvarious external devices connected to the vehicle 100. For example, theinterface 180 may include a port connectable to a mobile terminal andmay be connected to the mobile terminal through the port. In this case,the interface 180 can exchange data with the mobile terminal.

The interface 180 may serve as a passage through which electric energyis supplied to the mobile terminal connected thereto. When the mobileterminal is electrically connected to the interface 180, the interface180 can provide electric energy supplied from the power supply unit 190to the mobile terminal under the control of the controller 170.

The power supply unit 190 may provide power necessary for operations ofthe components of the vehicle 100 under the control of the controller170. The power supply unit 170 may be supplied with power from a battery(not shown) included in the vehicle.

The driver assistance apparatus 400 may assist a driver in driving thevehicle. The driver assistance apparatus 400 may include the camera 200.

The vehicle 200 may include a mono camera 200 a illustrated in FIGS. 3Ato 3C, and a stereo camera 200 b illustrated in FIGS. 3D to 3F.

The camera 200 may be referred to as a camera device for a vehicle.

FIG. 3A is a perspective view of a camera for a vehicle according to anembodiment of the present invention. FIG. 3B is an exploded perspectiveview of a camera for a vehicle according to an embodiment of the presentinvention. FIG. 3C is a side view of a camera for a vehicle, which iscut along line A-B shown in FIG. 3A, according to an embodiment of thepresent invention.

A camera 200 for a vehicle described with reference to FIGS. 3A to 3C isa single camera 200 a.

The camera 200 a may include a lens 211, an image sensor 214, and aprocessor 470.

According to an embodiment, the camera 200 a may further include aprocessing board 220, a light shield 230, a heat dissipation member 240,and a housing 250 individually or in combination.

Meanwhile, the housing 250 may include a first housing 251, a secondhousing 252, and a third housing 253.

The lens 211 may be housed in the lens housing in a manner in which thelens 211 is fastened by a nut 212 so as to be held in a hole 219 formedat one portion of the first housing 251.

The image sensor 214 may include at least one photoelectric conversiondevice capable of converting an optical signal into an electricalsignal. For example, the image sensor 214 may be a charge-coupled device(CCD) or a complimentary metal-oxide semiconductor (CMOS).

The image sensor 214 may be disposed at an appropriate position outsideor inside the vehicle in order to acquire images of the outside of thevehicle or images of the inside of the vehicle.

For example, the image sensor 214 may be disposed in proximity of afront windshield 10 in the interior of the vehicle in order to acquire afront field-of-view image of the vehicle.

Alternatively, the image sensor 214 may be disposed around a frontbumper or a radiator.

For example, the image sensor 214 may be disposed in proximity of a rearwindshield in the interior of the vehicle in order to acquire a rearfield-of-view image of the vehicle. Alternatively, the image sensor 214may be disposed around a rear bumper, a trunk, or a tailgate.

For example, the image sensor 214 may be disposed in proximity of atleast one side window in the interior of the vehicle in order to acquirea lateral field-of-view image of the vehicle.

Alternatively, the image sensor 214 may be disposed around a sidemirror, a fender, or a door.

The image sensor 214 may be disposed at rear of the lens 211 in order toacquire an image based on light introduced through the lens 211. Forexample, the image sensor 214 may be disposed vertical to the groundwith a predetermined distance spaced apart from the lens 211.

The processor 470 may be connected with the image sensor 214. Theprocessor 470 may computer-process an image acquired by the image sensor214. The processor 470 may control the image sensor 214.

The processor 470 may be implemented using at least one of applicationspecific integrated circuits (ASICs), digital signal processors (DSPs),digital signal processing devices (DSPDs), programmable logic devices(PLDs), field programmable gate arrays (FPGAs), processors, controllers,micro-controllers, microprocessors, or electronic units for performingother functions.

The processor 470 may be mounted on the processing board 220. Theprocessor 270 and a memory 440 may be mounted on the processing board220.

The processing board 220 may be disposed as being inclined in an overalllength direction. For example, a front surface or a rear surface of theprocessing board 220 may be disposed to face the front windshield 10.For example, the processing board 220 may be disposed in parallel withthe front windshield 10.

The front windshield 10 provided in a vehicle 100 may be generallyformed from a bonnet to a roof of the vehicle 100 to be inclined at apredetermined angle relative to the ground. In this case, the processingboard 220 is disposed to be inclined in the overall length direction,and hence, the camera 200 a may be formed smaller in size compared tothe case where the processing board 220 is disposed vertically orhorizontally. There is an advantage in that a more space in the vehicle100 may be secured as much as an amount by which the camera 200 a isreduced in size.

A plurality of devices or electronic components may be mounted on theprocessing board 220. In this case, heat may be generated due to theplurality of devices of components included in the processing board 220.

The processing board 220 may be disposed as being spaced apart from theimage sensor 241. Since the processing board 220 is disposed as beingspaced apart from the image sensor 241, the heat occurring in theprocessing board 220 may not affect performance of the image sensor 241.

The processing board 220 may be disposed at an optimum position suchthat the heat generated in the processing board 220 is prevented fromaffecting the image sensor 241. Specifically, the processing board 220may be disposed under the image sensor 241. Alternatively, theprocessing board 220 may be disposed at the front of the image sensor241.

One or more memories 440 may be mounted on the processing board 220. Amemory 440 may store images acquired by the image sensor 241, data ofvarious applications, data required to control the processor 470, ordata processed by the processor 470. Like the processor 470, the memory440 is one of major heat generating devices. With the processor 470disposed at the center of the processing board 220, the memory 440 maybe disposed around the processor 470. For example, one or memory 440 maybe disposed to surround the processor 470 positioned at the center. Inthis case, the processor 470 and the memories 440, which are heatgenerating devices, may be disposed at positions most distal from theimage sensor 241.

The processor 470 may be electrically connected with a controller 170.The processor 470 may be controlled by the controller 170.

The light shield 230 may be disposed at the front of the lens 211. Thelight shield 230 may shield light unnecessary to acquire an image suchthat the light is not introduced into the lens 211. For example, thelight shield 230 may shield light reflected from the windshield 10, adashboard of the vehicle, or the like. In addition, the light shield 230may shield light generated by an unnecessary light source.

The light shield 230 may have a partition structure. For example, thelight shield 230 may have a lower partition structure. Meanwhile, theshape of the light shield 230 may vary depending on a vehicle model. Forexample, a curvature of the windshield and an angle formed by thewindshield and the ground may vary depending on a vehicle model, andhence, the light shield 230 may have a shape corresponding to a vehiclemodel on which the camera 200 a is installed. To this end, the lightshield 230 may have a detachable structure.

The heat dissipation member 240 may be disposed at the rear end of theimage sensor 214. The heat dissipation member 240 may contact the heatdissipation member 240 or an image sensor board on which the imagesensor 214 is mounted. The heat dissipation member 240 may handle heatoccurring in the image sensor 214.

As described above, the image sensor 241 is sensitive to heat. The heatdissipation member 240 may be disposed between the image sensor 214 andthe third housing 253. The heat dissipation member 240 may be disposedto contact the image sensor 214 and the third housing 253. In this case,the heat dissipation member 240 may dissipate heat through the thirdhousing 253.

For example, the heat dissipation member 240 may be any one of a thermalpad and a thermal grease.

The housing 250 may include the lens housing 217, the first housing 251,the second housing 252, and the third housing 253.

The lens housing 217 may accommodate at least one lens 211 and protectthe lens 211 from an external impact.

The first housing 251 may be formed to surround the image sensor 241.The first housing 251 may include the hole 219. While being accommodatedin the lens housing and held in the hole 219, the lens 211 may beconnected with the image sensor 214.

The first housing 251 may be formed to increase in thickness in adirection toward the image sensor 214. For example, the first housing251 may be formed by die casting. In this case, in order to preventdegradation of performance of the image sensor 214 caused by heat, aportion of the first housing 251 in proximity to the image sensor 214may be formed thicker than other portions.

The first housing may be formed with a thickness greater than athickness of the third housing 253. If a housing has a great thickness,heat is delivered slowly. Thus, in the case where the first housing 251is formed with a thickness greater than a thickness of the third housing253, heat generated in the camera 200 a may be dissipated through thethird housing 253 rather than the first housing 251 which is disposed inproximity to the front windshield such that heat dissipation is hard tobe carried out.

Meanwhile, according to an embodiment, the lens housing 217 and thefirst housing 251 may be integrally formed as one body.

The second housing 252 may be disposed at the front end of theprocessing board 220. The second housing 252 may be fastened to thefirst housing 251 and the third housing 253 using a predeterminedfastening means.

The second housing 252 may include an attachment means to allow thelight shield to be attached thereto. The light shield 230 may beattached to the second housing 252 using the attachment means.

The first and second housing 252 and 253 may be formed of a syntheticresin material.

The third housing 253 may be fastened to the first housing 251 and thesecond housing 252 using a predetermined fastening means. According toan embodiment, the first to third housing 251, 252, and 253 may beintegrally formed as one body.

The third housing 253 may be formed to surround the processing board220. The third housing 253 may be disposed at the rear or lower end ofthe processing board 220. The third housing 253 may be formed of athermally conductive material. For example, the third housing 253 may beformed of a metal such as aluminum. Since the third housing 253 isformed of a thermally conductive material, heat may be dissipatedefficiently.

In the case where the first and second housings 251 and 252 are formedof a synthetic resin material and the third housing 253 is formed of athermally conductive material, heat inside the camera for the vehiclemay be dissipated through the third housing 253 rather than the firstand second housings 251 and 252. That is, in the case where the camera200 a is mounted on a windshield, the first and second housings 251 and252 may be disposed in proximity to the windshield and hence heat cannotbe dissipated through the first and second housings 251 and 252. In thiscase, the heat may be efficiently dissipated through the third housing253.

Meanwhile, in the case where the third housing 253 is formed ofaluminum, it may be advantageous in protecting internal components(e.g., the image sensor 241 and the processor 470) from electro-magneticcompatibility (EMC) and electrostatic discharge (ESC).

The third housing 253 may contact the processing board 220. In thiscase, the third housing 253 may transfer heat through a portion thereofin contact with the processing board 220, thereby efficientlydissipating the heat to an outside.

The third housing 253 may further include a heat dissipation unit 291.For example, the heat dissipation unit 291 may include at least one of aheat sink, a heat dissipation fin, a thermal pad, or a thermal grease.

The heat dissipation unit 291 may dissipate heat generated in the camera200 a to an outside. For example, the heat dissipation unit 291 may bedisposed between the processing board 220 and the third housing 253. Theheat dissipation unit 291 may contact the processing board 220 and thethird housing 253 to dissipate heat generated in the interior of theprocessing board 220.

The third housing 253 may further include an air outlet. The air outletis a hole for dissipating high-temperature air in the camera 200 a tothe outside of the camera 200 a. The camera 200 a may include an airflow passage connected to the air outlet. The air flow passage may guidehigh-temperature air in the camera 200 a toward the air outlet.

The camera 200 a may further include a moisture-proof unit. Themoisture-proof unit may be formed in the shape of patch attached to theair outlet. The moisture-proof unit may be a moisture-proof unit formedof Gore-Tex material. The moisture-proof unit may discharge humid airinside the camera 200 a to the outside. In addition, the moisture-proofunit may prevent humid air outside the camera 200 a from coming inside.

FIG. 3D is a perspective view of a camera for a vehicle according to anembodiment of the present invention. FIG. 3E is an exploded perspectiveview of a camera for a vehicle according to an embodiment of the presentinvention. FIG. 3F is a side view of a camera for a vehicle, which iscut along line C-D shown in FIG. 3D, according to an embodiment of thepresent invention.

A camera 200 for a vehicle described with reference to FIGS. 3D to 3F isa stereo camera 200 b.

Description about the single camera 200 a with reference to FIGS. 3A to3C may apply to the stereo camera 200 b. That is, a first camera and asecond camera included in the stereo camera 200 b may be the cameradescribed with reference to FIGS. 3A to 3C. The stereo camera 200 b mayinclude a first lens 211 a, a second lens 211 b, a first image sensor214 a, a second image sensor 214 b, and a processor 470 a.

According to an embodiment, the camera 200 b may further include aprocessing board 220 a, a first light shield 230 a, a second lightshield 230 b, and a housing 250 a individually or in combination.

Meanwhile, the housing may include a first lens housing 217 a, a secondlens housing 217 b, a first housing 251 a, a second housing 252 a, and athird housing 253 a.

Description about the lens 211 illustrated in FIGS. 3A to 3C may applyto the first lens 211 a and the second lens 211 b.

Description about the image sensor 214 illustrated in FIGS. 3A to 3C mayapply to the first image sensor 214 a and the second image sensor 214 b.

Meanwhile, a module including the first lens 211 a and the first imagesensor 214 a may be referred to as a first image acquisition module. Inaddition, a module including the second lens 211 b and the second imagesensor 214 b may be referred to as a second image acquisition module.

The processor 470 a may be electrically connected with the first imagesensor 214 a and the second image sensor 214 b. The processor 470 maycomputer-process images processed by the first image sensor 214 a andthe second image sensor 214 b. In this case, the processor 470 may forma disparity map or calculate a disparity based on the images acquired bythe first image sensor 214 a and the second image sensor 214 b.

The processor 470 a may be implemented using at least one of applicationspecific integrated circuits (ASICs), digital signal processors (DSPs),digital signal processing devices (DSPDs), programmable logic devices(PLDs), field programmable gate arrays (FPGAs), processors, controllers,micro-controllers, microprocessors, or electronic units for performingother functions.

The processor 470 a may be mounted on the processing board 220 a.Description about the processing board 220 illustrated in FIGS. 3A to 3Cmay apply to the processing board 220 a.

Description about the light shield 230 illustrated in FIGS. 3 to 5 mayapply to the first light shield 230 a and the second light shield 230 b.

Description about the lens housing 217 illustrated in FIGS. 3 to 5 mayapply to the first lens housing 217 a and the second lens housing 217 b.

Description about the first housing 251 illustrated in FIGS. 3A to 3Cmay apply to the first housing 251 a.

Description about the second housing 252 illustrated in FIGS. 3A to 3Cmay apply to the second housing 252 a.

Description about the third housing 253 illustrated in FIGS. 3A to 3Cmay apply to the third housing 253 a.

FIG. 4A is a block diagram illustrating a driver assistance apparatusaccording to an embodiment of the present invention.

Referring to FIG. 4A, a driver assistance apparatus 400 may include acamera 200 for a vehicle, a processor 470, an interface unit 430, amemory 440, and an output unit 450,

According to an embodiment, the driver assistance apparatus 400 mayfurther include a communication unit 410, an input unit 420, and a powersupply 440 individually or in combination.

According to an embodiment, unlike the example shown in FIG. 4A, theprocessor 470, the interface unit 430, and the memory 440 may beclassified as subordinate components of the camera 200. In this case,the camera 200 may function as the driver assistance apparatus 200.

The camera 200 may be mounted at one portion of a vehicle 100 andacquire an image.

For example, the camera 200 may be disposed in proximity of a frontwindshield 10 in the interior of the vehicle in order to acquire a frontfield-of-view image of the vehicle. Alternatively, the camera 200 may bedisposed around a front bumper or a radiator grill.

For example, the camera 200 may be disposed in proximity to a rearwindshield in the interior of the vehicle in order to acquire a rearfield-of-view image of the vehicle. Alternatively, the camera 200 may bedisposed around a rear bumper, a trunk, or a tailgate.

For example, the camera 200 may be disposed in proximity to at least oneside window in the interior of the vehicle in order to acquire a lateralfield-of-view image of the vehicle.

Alternatively, the camera 200 may be disposed around a side mirror, afender, or a door.

The camera 200 may include an image sensor 214 and an actuator 401.

The image sensor 214 is the same as described with reference to FIGS. 3Ato 3F.

According to an embodiment, the camera 200 may be the stereo camera 200b (see FIGS. 3D to 3F).

In the case where the camera 200 is the stereo camera 200 b, the camera200 may include a first camera, a second camera, and the processor 470.

The interface unit 430 may receive a variety of signals, information, ordata. The interface unit 430 may transmit a signal, information, or dataprocessed or generated by the processor 470. To this end, the interfaceunit 430 may perform data communication with a controller 170, a displaydevice 141 for the vehicle, a sensing unit 125, and a vehicle drive unit150 in the vehicle by a wireless or wired communication method.

The interface unit 430 may receive sensor information from thecontroller 170 or the sensing unit 125.

Here, the sensor information may include at least one of the following:vehicle heading information, vehicle location information (GlobalPositioning System (GPS) information), vehicle angle information,vehicle speed information, vehicle steering information, vehicleacceleration information, vehicle tilt information, vehicledrive/reverse information, battery information, fuel information, tireinformation, vehicle lamp information (e.g., turn signal information),in-vehicle temperature information, in-vehicle humidity information, andinformation on whether or not it is raining.

The sensor information may be acquired from a heading sensor, a yawsensor, a gyro sensor, a position module, a vehicle drive/reversesensor, a wheel sensor, a vehicle speed sensor, a steering angle sensor,a vehicle tilt sensor, a battery sensor, a fuel sensor, a tire sensor, asensor for sensing steering in response to rotation of a steering wheel,in-vehicle temperature sensor, in-vehicle humidity sensor, a rainsensor, etc. Meanwhile, the position module may include a GPS module forreceiving GPS information.

The interface unit 430 may receive navigation information through datacommunication with the controller 170, the display device 141, or anadditional navigation device. Here, the navigation information mayinclude travel road information, travel lane information, destinationinformation, route information, heading information, map information,and information on the current location of the vehicle. Meanwhile, thenavigation information may include a location of the vehicle on a road.

The interface unit 430 may provide a signal to the controller 170 or thevehicle drive unit 150. Here, the signal may be a control signal.

For example, the interface unit 430 may communicate with a power sourcedriver 151 which controls a power source. The interface unit 430 mayprovide a signal, generated by the processor 470, to the power sourcedriver 151.

For example, the interface unit 430 may communicate with a brake driver153 which controls a brake device. The interface unit 430 may provide asignal, generated by the processor 470, to the brake driver 153.

For example, the interface unit 430 may communicate with a steeringdriver 152 which controls a steering device. The interface unit 430 mayprovide a signal, generated by the processor 470, to the steering driver152.

The memory 440 may store a variety of data required for overalloperations of the driver assistance apparatus 400, such as programs forprocessing or controlling of the processor 470.

The memory 440 may be a variety of storage devices, such as a ROM, aRAM, an EPROM, a flash drive, a hard drive, etc. According to anembodiment, the memory 440 may be a subordinate component of theprocessor 470.

The memory 440 may store a priority order of a plurality of trafficsigns. The priority order of the plurality of traffic signs may bestored in the memory 440 as a default value.

The memory 440 may store the priority order in which a traffic lampsignal has a priority level higher than that of a traffic signal markedon a road surface or a traffic signal marked on a traffic signboard.

The memory 440 may store the priority order in which a hand signal has apriority level higher than that of a traffic lamp signal.

The memory 440 may store the priority order in which a stop signalmarked on a traffic signboard is at the highest priority level.

The memory 440 may store a priority order of a plurality of items oftravel information. The priority order of the plurality of items oftravel information may be stored in the memory 440 as a default value.

The memory 440 may store the priority order in which turn signalinformation has a priority level higher than that of destinationinformation.

The processor 470 may be electrically connected with each unit of thedriver assistance apparatus 400.

The processor 470 may control overall operations of each unit of thedriver assistance apparatus 400.

The processor 470 may receive a vehicle external image photographed bythe camera 200. Here, the vehicle external image may be a frontfield-of-view image of the vehicle.

The processor 470 may detect at least one traffic sign based on avehicle external image.

For example, based on a vehicle external image, the processor 470 maydetect at least one of a traffic signal marked on a road surface, atraffic sign marked on a traffic signboard, a traffic lamp signal, aconstruction zone warning signal, a traffic signal attached to anothervehicle, or a hand signal.

The processor 470 may acquire a traffic sign by acquiring a drawing ortext marked on a road surface based on a vehicle external image.

The processor 470 may acquire a traffic sign by detecting a drawing ortext marked on a traffic signboard based on a vehicle external image.

The processor 470 may acquire a traffic sign by detecting a color orposition of light output from a traffic lamp based on a vehicle externalimage.

The processor 470 may acquire a traffic sign by detecting at least oneof a traffic corn, a temporary traffic signboard, a traffic baton, or atraffic guide light device based on a vehicle external image.

The processor 470 may acquire a traffic sign by detecting anothervehicle based on a vehicle external image and detecting a traffic signattached to a body, a bumper, or a glass of the detected vehicle.

The processor 470 may acquire a traffic sign by detecting a hand signalof a traffic police officer based on a vehicle external image. Forexample, the processor 470 may detect characteristics of traffic policeofficer cap and uniform and compare the detected characteristics withdata pre-stored in the memory 440, thereby detecting a traffic policeofficer. When the traffic police officer is detected, the processor 470may detect a predetermined motion of the traffic police officer andcompare the detected motion with the data pre-stored in the memory 440,thereby detecting a hand signal.

The processor 470 may acquire travel information of the vehicle. Theprocessor 470 may detect travel information of the vehicle based on avehicle external image.

For example, the processor 470 may acquire travel road information,travel lane information, etc. based on a vehicle external image. Theprocessor 470 may acquire the travel road information by detecting anobject (e.g., a tall gate, a direction sign, and a traffic signboard)indicating entrance to a road. The processor 470 may acquire the travellane information by detecting a lane from the vehicle external image.

The processor 470 may acquire travel information of the vehicle from thecommunication unit 110, the input unit 120, the sensing unit 125, thememory 130, the display device 141, the navigation device, or thecontroller 170 through the interface unit 430.

For example, the processor 470 may acquire travel information of thevehicle by receiving navigation information from the navigation deviceor the display device 141. Here, the navigation information may includetravel road information, travel lane information, destinationinformation, route information, heading information, map information,and information on the current location of the vehicle.

For example, the processor 470 may receive turn signal information fromthe input unit 120 or the controller 170.

For example, the processor 470 may receive steering information orheading information from the sensing unit 125 or the controller 170.

For example, the processor 470 may receive vehicle model informationfrom the memory 130.

The processor 470 may determine whether a traffic sign fits travelinformation of the vehicle.

For example, the processor 470 may determine whether a traffic sign fitsat least one of travel road information, travel lane information,destination information, route information, turn signal information,steering information, heading information, or vehicle model information.

When the traffic sign fits the travel information, the processor 470 mayperform control to output the traffic sign through the output unit 450.

The processor 470 may perform control to display a graphic imagecorresponding to the traffic sign through a display unit 451.

The processor 470 may perform control to output sound corresponding tothe traffic sign through a sound output unit 452.

When the traffic sign does not fit the travel information, the processor470 may perform control not to output the traffic sign. As such, as atraffic sign fitting travel information of the vehicle is output and atraffic sign not fitting the travel information is not output, only atraffic sign fitting a travelling situation is allowed to be provided toa driver. Hence, it is possible to prevent confusion to a driver andprovide only correct information.

When a traffic sign fits the travel information of the vehicle, theprocessor 470 may provide at least one of a steering control signal, anacceleration control signal, or a brake control signal to the vehicledrive unit 150 based on the traffic sign through the interface unit 430.

For example, the processor 470 may provide a brake control signal to thebrake driver 153 based on a traffic sign indicative of speed limitinformation.

For example, the processor 470 may provide an acceleration controlsignal to the power source driver 151 based on a traffic sign indicativeof uphill information.

For example, the processor 470 may provide a steering controlinformation to the steering driver 152 based on a traffic signindicative of curve information.

Meanwhile, a traffic sign may include at least one of a traffic signalmarked on a road surface, a traffic signal marked on a trafficsignboard, a traffic lamp signal, a construction zone warning signal, atraffic signal attached to another vehicle, and a hand signal.

The traffic signal marked on a road surface may be a drawing or textmarked on the road surface.

The traffic signal marked on the traffic signboard may be a drawing ortext marked on the traffic signboard.

The traffic lamp signal may be a color or position of light output fromthe traffic lamp.

The construction zone warning signal may include at least one of atraffic corn, a temporary traffic signboard, a traffic baton, or a guidelight device.

The traffic signal attached to another vehicle may be a traffic signattached to a body, a bumper, or a glass of another vehicle.

The hand signal may be a hand signal by a traffic police officer.

When a plurality of traffic signs fits travel information, the processor470 may perform control to output a traffic sign having a higherpriority level among the plurality of traffic signs according to apriority order. Here, the priority order may be a priority order of theplurality of traffic signs. Here, the plurality of traffic signs mayinclude items of information contradictory to each other. The processor470 may perform control not to output a traffic sign having a lowerpriority level.

Meanwhile, travel information of the vehicle may include at least one oftravel road information, travel lane information, destinationinformation, route information, turn signal information, steeringinformation, heading information, or vehicle model information.

When a plurality of items of travel information is acquired, theprocessor 470 may determine, based on a priority order, whether atraffic sign fits an item of travel information having a higher prioritylevel among the plurality of items of traffic information. Here, thepriority order may be a priority order of a plurality of items of travelinformation stored in the memory 440. Here, the plurality of items oftravel information may include items of information contradictory toeach other.

For example, when turn signal information and destination information donot fit each other, the processor 470 may determine whether a trafficsign having a priority level higher than that of the destinationinformation fits the turn signal information, wherein the priority levelis stored in the memory 440. In this case, the processor 470 may displaya traffic signal according to a driver's intention.

The processor 470 may detect a plurality of traffic signs based on afront field-of-view image of the vehicle.

The processor 470 may calculate a distance between the vehicle 100 andeach of the plurality of traffic signs. The processor 470 may calculatea distance between the vehicle 100 and each of the plurality of trafficsigns using an image distance detection algorithm. For example, when astereo image is acquired, the processor 470 may calculate the distanceby calculating a disparity. For example, the processor 470 may calculatethe distance using a pinhole algorithm. For example, the processor 470may calculate the distance using an actual distance and a pixeldistance. For example, the processor 470 may calculate the distanceusing a motion vector of an object. For example, the processor maycalculate the distance using a variation of size of an object per pixel.

The processor 470 may perform control to output a traffic sign closestto the vehicle 100 among a plurality of traffic signs fitting travelinformation.

The processor 470 may detect a first traffic sign and a second trafficsign based on a vehicle external image.

When the first traffic sign fits travel information and the secondtraffic sign does not fit the travel information, the processor 470 mayperform control such that the first traffic sign is output through theoutput unit 450 whereas the second traffic sign is not output.

The processor 470 may receive state information of the vehicle based ona driving manipulation signal received through a driving manipulationdevice 121 through the interface unit 430. Here, the state informationof the may include acceleration state information, steering stateinformation, and brake state information.

The processor 470 may determine whether state information of the vehiclebased on a driving manipulation signal fits output traffic signinformation.

When the state information of the vehicle does not fit traffic signinformation, the processor 470 may output an alarm through the outputunit 450.

For example, in the case where output traffic sign information is speedlimit information, when state information of the vehicle indicates astate in which the vehicle is accelerating beyond the speed limit, theprocessor 470 may output an alarm through the output unit 450.

The processor 470 may acquire rampway entering situation information.For example, the processor 470 may acquire rampway entering situationinformation by receiving navigation information.

For example, the processor 470 may acquire rampway entering situationinformation by detecting a road or a traffic signboard indicative of arampway from a vehicle external image.

The processor 470 may perform control to output a traffic sign whichfits travel information among a plurality of traffic signs placed on aplurality of paths forming a rampway.

The processor 470 may detect a plurality of traffic lamps based on avehicle external image. The processor 470 may determine whether theplurality of traffic lamps fits travel information.

The processor 470 may perform control to output a traffic sign generatedby a traffic lamp which is closest to the vehicle 100 among a pluralityof traffic lamps fitting travel information.

The processor 470 may acquire lane information on a lane in which thevehicle 100 is currently traveling. For example, the processor 470 mayacquire the lane information on a lane in which the vehicle 100 iscurrently traveling, by receiving navigation information. For example,the processor 470 may acquire the lane information on a lane in whichthe vehicle 100 is currently traveling, by detecting the lane from avehicle external image.

The processor 470 may detect a plurality of traffic signs based on afront field-of-view image of the vehicle. The processor 470 may performcontrol to output a traffic sign which fits lane information among theplurality of traffic signs.

The plurality of traffic signs may include a first traffic sign based ona traffic signboard for a permitted left turn of the vehicle 100, and asecond traffic sign based on a traffic lamp for straight running of thevehicle 100.

Based on a front field-of-view image of the vehicle, the processor 470may detect the first traffic sign based on a traffic signboard for apermitted left turn of the vehicle 100, and the second traffic signbased on a traffic signal for straight running of the vehicle 100.

The processor 470 may acquire travel situation information indicating asituation that the vehicle 100 is traveling in a left-turn lane. Forexample, the processor 470 may acquire the travel situation informationindicating a situation that the vehicle 100 is traveling in a left-turnlane, by receiving navigation information. For example, the processor470 may acquire the travel situation information indicating a situationthat the vehicle 10 is traveling in a left-turn lane, by detecting thelane from a vehicle external image.

The processor 470 may perform control such that the first traffic signfitting the travel situation information is output whereas the secondtraffic sign not fitting the travel situation information is not output.

A plurality of traffic signs may include a first traffic sign for aright turn of the vehicle 100, and a second traffic sign based on asecond traffic lamp for straight running of the vehicle 100.

Based on a front field-of-view image of the vehicle, the processor 470may detect both the first traffic sign for a right turn of the vehicle100, and the second traffic sign based on a second traffic lamp forstraight running of the vehicle 100.

The processor may acquire travel situation information indicating asituation that the vehicle 100 is traveling in a right-turn lane. Forexample, the processor 470 may acquire the travel situation informationindicating a situation that the vehicle 100 is traveling in a right-turnlane, by receiving navigation information. For example, the processor470 may acquire the travel situation information indicating a situationthat the vehicle 100 is traveling in a right-turn lane, by detecting thelane from a vehicle external image.

The processor 470 may perform control such that the first traffic signfitting the travel situation information is output whereas the secondtraffic sign not fitting the travel situation information is not output.

The processor 470 may acquire vehicle speed information. When a speed ofthe vehicle 100 is equal to or lower than a reference speed, theprocessor 470 may perform control not to output a traffic sign even inthe case where the traffic sign fits travel information.

FIG. 4B is a diagram illustrating internal configurations of a processorand a signal processing procedure by each configuration according to anembodiment of the present invention.

Referring to FIG. 4B, the processor 470 may include an image processor471, a determiner 474, and a signal provider 477.

The image processor 471 may receive an image from the camera 200.

The image processor 471 may computer-process the received image usingvarious techniques.

The image processor 471 may detect an object based on the receivedimage.

The image processor 471 may detect at least one traffic sign based onthe received image.

The image processor 471 may calculate a distance between the vehicle 100and the traffic sign.

The image processor 471 may calculate the distance between the vehicle100 and the traffic sign using an image distance detection algorithm.For example, when a stereo image is acquired, the image processor 471may calculate the distance by calculating a disparity.

For example, the image processor 471 may calculate the distance using apinhole model algorithm. For example, the image processor 471 maycalculate the distance using an actual distance and a pixel distance.For example, the image processor 471 may calculate the distance using amotion vector of an object. For example, the image processor 471 maycalculate the distance using a variation of size of an object per pixel.

Based on a received image, the image processor 471 may detect a road ora traffic signboard indicative of a rampway.

Based on a received image, the image processor 471 may detect a lane inwhich the vehicle 100 is currently traveling.

Based on a received image, the image processor 471 may detect aleft-turn lane or a right-turn lane.

The determiner 474 may make a determination based on informationreceived from the image processor 471, information received from theinterface unit 430, or data received from the memory 440.

The determiner 474 may acquire travel information of the vehicle. Forexample, the determiner 474 may receive travel information of thevehicle from the image processor 471. For example, the determiner 474acquire travel information of the vehicle from the communication unit110, the input unit 120, the sensing unit 125, the memory 130, thedisplay device 141, the navigation device, or the controller 170 throughthe interface unit 430.

The determiner 474 may determine whether a traffic sign fits travelinformation of the vehicle. For example, the determiner 474 maydetermine whether the traffic sign fits at least one of travel roadinformation, travel lane information, destination information, routeinformation, turn signal information, steering information, headinginformation, or vehicle model information.

The determiner 474 may receive information on a priority order of aplurality of traffic signs from the memory 440. Based on the informationon the priority order, the determiner 474 may determine a traffic signhaving a higher priority level among the plurality of traffic signs.

The determiner 474 may receive information on a priority order of aplurality of items of travel information from the memory 440. Based onthe information on the priority order, the determiner 474 may determinean item of travel information having a higher priority level among theplurality of items of travel information.

Based on a determination by the determiner 474, the signal provider 477may provide a control signal to the output unit 450. Specifically, basedon a determination by the determiner 474, the signal provider 477 mayprovide a control signal to the output unit 450 to output a trafficsign.

The signal provider 477 may provide a control signal to the output unit450 so as to output a traffic sign fitting the travel information of thevehicle.

The signal provider 477 may provide a control signal to the display unit451 so as to display a graphic image corresponding to the traffic sign.

The signal provider 477 may provide a control signal to the sound outputunit 452 so as to output sound corresponding to the traffic sign.

When the traffic sign fits the travel information of the vehicle, thesignal provider 477 may provide a control signal to the controller 170or the vehicle drive unit 150 based on the traffic sign through theinterface unit 430. Specifically, the signal provider 477 may provide atleast one of a brake control signal, an acceleration control signal, ora steering control signal.

Referring back to FIG. 4A, the output unit 450 may output data orinformation, processed by the processor 470, in accordance with controlby the processor 470.

The output unit 450 may include the display unit 451 and the soundoutput unit 452.

The display unit 451 may display information processed by the processor470. The display unit 451 may display an image relating to an operationof the driver assistance apparatus 400. In order to display such animage, the display unit 451 may include a cluster or a Head Up Display(HUD) disposed at the front in the interior of the vehicle. Meanwhile,when the display unit 451 is the HUD, the display unit 451 may include aprojection module which projects an image onto the front windshield 10or a combiner of the vehicle 100.

The sound output unit 452 may output sound to an outside based on anaudio signal processed by the processor 470. To this end, the soundoutput unit 452 may include at least one speaker.

The communication unit 410 may exchange data with a different devicelocated inside or outside the vehicle in a wireless method. Here, thedifferent device may include a mobile terminal, a server, or anothervehicle.

For example, the communication unit 410 may wirelessly exchange datawith a mobile terminal of a driver of the vehicle.

As a method for the wireless data communication, a variety of methodsare allowed, such as Bluetooth, WiFi Direct, WiFi, APiX, and NFC.

For example, the communication unit 410 may receive weather informationand road traffic information such as Transport Protocol Expert Group(TPEG) information from a mobile terminal or a server.

Meanwhile, when a user is onboard the vehicle, the user's mobileterminal and the driver assistance apparatus 200 may be pairedautomatically or according to execution of an application by the user.

The communication unit 410 may receive traffic lamp change informationfrom an external server. Here, the external server may be a serverlocated in a traffic control.

The input unit 420 may receive a user input. The input unit 420 mayinclude a mechanical input device, a touch input device, a voice inputdevice, or a wireless input device.

The mechanical input device may include a button, a lever, a job wheel,a switch, etc.

The touch input device may include at least one touch sensor.

The touch input device may be implemented as a touch screen.

The voice input device may include a microphone which converts a user'svoice into an electrical signal.

The wireless input device may receive a wireless user input which isinput using a key from the outside of the vehicle 100.

The input unit 420 may receive a user input for opening or closing adoor included in the vehicle 100.

The power supply 490 may supply power required to operate each componentunder the control of the processor 470. The power supply 490 may besupplied with power from a battery or the like provided in the vehicle.

FIGS. 5A and 5B illustrate various examples of an internal block diagramof the image processor illustrated in FIG. 4B, and FIGS. 5C and 5D arediagrams illustrating operation of the processor illustrated in FIG. 5B.

Referring to FIG. 5A, FIG. 5A is an example of an internal block diagramof the image processor 471, and the image processor 471 may include animage preprocessor 501, a disparity calculator 502, an object detector504, an object tracking unit 506, and an application unit 507.

The image preprocessor 501 may receive an image from the camera 200 andperform preprocessing with respect to the received image.

Specifically, the image preprocessor 501 may perform noise reduction,rectification, calibration, color enhancement, color space conversion(CSC), interpolation, camera gain control, etc. with respect to animage. Accordingly, an image more vivid than a stereo image acquired bythe camera 200 may be acquired.

The disparity calculator 502 may receive images signal-processed by theimage preprocessor 501, perform stereo matching with respect to thereceived images, and acquire a disparity map according to the stereomatching. That is, disparity information on a stereo image may beacquired.

Here, the stereo matching may be performed on a pixel unit or apredetermined block unit of stereo images. Meanwhile, the disparity mapmay refer to a map that numerically indicates binocular parallaxinformation of a stereo image, that is, left and right images.

A segmentation unit 503 may perform segmentation or clustering on atleast one of images based on disparity information received from thedisparity calculator 502.

Specifically, the segmentation unit 503 may divide at least one ofstereo images into a background and a foreground based on the disparityinformation.

For example, a region with disparity information equal to or less than apredetermined level in the disparity map may be calculated as thebackground, and the corresponding region may be excluded. As such, therelative foreground may be separated.

In another example, a region with disparity information equal to orgreater than the predetermined level in the disparity map may becalculated as the foreground, and the corresponding region may beextracted. As such, the foreground may be separated.

As such, the foreground and the background may be separated based ondisparity information extracted based on a stereo image, and thus,signal processing speed, signal processing amount, and so on may bereduced during subsequent object detection.

Then, the object detector 504 may detect an object based on imagesegmentation by the segmentation unit 503.

That is, the object detector 504 may detect an object in at least one ofimages based on the disparity information.

Specifically, the object detector 504 may detect an object in at leastone of images. For example, an object may be detected from a foregroundextracted via image segmentation.

Then, an object verification unit 505 may classify and verify aseparated object.

To this end, the object verification unit 505 may employ anidentification scheme using a neural network, a support vector machine(SVM) scheme, an identification scheme via AdaBoost using Haar-likefeatures, a histograms of oriented gradients (HOG) scheme, or the like.

Meanwhile, the object verification unit 505 may compare objects storedin the memory 440 with the detected object to verify the detectedobject.

For example, the object verification unit 505 may verify a surroundingvehicle, a lane, a road surface, a signboard, a dangerous area, atunnel, or the like, positioned around a vehicle.

The object tracking unit 506 may track the verified object. For example,the object tracking unit 506 may verify an object in sequentiallyacquired images, calculate motion of the verified object or a motionvector, and track movement of the corresponding object based on thecalculated motion or motion vector.

Accordingly, a surrounding vehicle, a lane, a road surface, a signboard,a dangerous area, a tunnel, or the like, positioned around a vehicle,may be tracked.

Then, the application unit 507 may calculate a risk level of the vehicle100 based on various objects located around the vehicle, e.g., anothervehicle, a lane, a road surface, a signboard, etc. In addition, apossibility of crash with a vehicle ahead, occurrence of slipping of thevehicle, etc. may be calculated.

In addition, as driver assistant information, the application unit 507may output a message or the like for informing a user of the calculatedrisk level, the possibility of crash, or occurrence of slipping.Alternatively, a control signal for controlling a position or travellingof the vehicle 100 may be generated as vehicle control information.

Meanwhile, according to an embodiment, the processor 470 may includeonly some of the image processor 501, the disparity calculator 502, thesegmentation unit 503, the object detector 504, the object verificationunit 505, the object tracking unit 506, and the application unit 507.For example, when the camera 200 is implemented as a mono camera, thedisparity calculator 502 may be excluded. In addition, according to anembodiment, the segmentation unit 503 may be excluded.

FIG. 5B is another example of an internal block diagram of a processor.

Referring to the drawing, a processor 470 of FIG. 5B has the sameinternal constituent units as the processor 470 of FIG. 4A but isdifferent in terms of a signal processing order. Hereinafter,embodiments of the present invention will be described in terms of suchdifference.

An object detector 504 may receive stereo images, and detect an objectfrom at least one of the stereo images. Unlike FIG. 5A, the object maybe detected directly from a stereo image, rather than an image which issegmented based on disparity information.

Then, an object verification unit 505 may classify and verify an objectbased on an image segment received from a segmentation unit 503 and anobject detected by the object detector 504.

To this end, the object verification unit 505 may employ anidentification scheme using a neural network, a support vector machine(SVM) scheme, an identification scheme via AdaBoost using Haar-likefeatures, a histograms of oriented gradients (HOG) scheme, or the like.

FIGS. 5C and 5D are diagrams illustrating an operating method of theprocessor 470, illustrated in FIGS. 5A and 5B, based on stereo imagesrespectively acquired in first and second frame sections.

Referring to FIG. 5, the stereo camera 200 b acquires a stereo imageduring the first frame section.

The disparity calculator 502 in the processor 470 receives stereo imagesFR1 a and FR1 b signal-processed by the image preprocessor 501, andperforms stereo matching on the received stereo images FR1 a and FR1 bto acquire a disparity map 520.

The disparity map 520 may be obtained by leveling binocular parallaxbetween the stereo images FR1 a and FR1 b, and calculation may beperformed in such a way that a distance from a vehicle is calculated tobe close, as a disparity level is increased, and the distance from thevehicle is calculated to be far as a disparity level is reduced.

Meanwhile, when the disparity map is displayed, brightness may be highas a disparity level is increased and, brightness may be low as adisparity level is reduced.

The drawing shows an example in which first to fourth lanes 528 a, 528b, 528 c, and 528 d, and the like have respective disparity levelscorresponding thereto and a construction zone 522, a first precedingvehicle 524, and a second preceding vehicle 526 have respectivedisparity levels corresponding thereto in the disparity map 520.

The segmentation unit 503, the object detector 504, and the objectverification unit 505 may perform segmentation, object detection, andobject verification, respectively, on at least one of the stereo imagesFR1 a and FR1 b based on the disparity map 520.

The drawing shows an example in which objection detection andverification are performed on the second stereo image FR1 b using thedisparity map 520.

That is, an object, for example, first to fourth lanes 538 a, 538 b, 538c, and 538 d, a construction zone 532, a first preceding vehicle 534,and a second preceding vehicle 536 may be detected and verified in animage 530.

Then, referring to FIG. 5D, the stereo camera 200 b acquires a stereoimage during the second frame section.

The disparity calculator 502 in the processor 470 receives stereo imagesFR2 a and FR2 b signal-processed by the image preprocessor 501, andperforms stereo matching on the received stereo images FR2 a and FR2 bto acquire a disparity map 540.

The drawing shows an example in which first to fourth lanes 548 a, 548b, 548 c, and 548 d, and the like have respective disparity levelscorresponding thereto and a construction zone 542, a first precedingvehicle 544, and a second preceding vehicle 546 have respectivedisparity levels corresponding thereto in the disparity map 540.

The segmentation unit 503, the object detector 504, and the objectverification unit 505 may perform segmentation, object detection, andobject verification, respectively, on at least one of the stereo imagesFR2 a and FR2 b based on the disparity map 540.

The drawing shows an example in which objection detection andverification are performed on the second stereo image FR2 b using thedisparity map 540.

That is, an object, for example, first to fourth lanes 558 a, 558 b, 558c, and 558 d, a construction zone 552, a first preceding vehicle 554,and a second preceding vehicle 556 may be detected and verified in animage 550.

Meanwhile, the object tracking unit 506 may track a verified object bycomparing FIGS. 5A and 5B.

Specifically, the object tracking unit 506 may track movement of eachobject verified in FIGS. 5A and 5B based on motion of a correspondingobject or a motion vector thereof. Accordingly, a lane, a constructionzone, a first preceding vehicle, a second preceding vehicle, or thelike, positioned around a vehicle, may be tracked.

FIGS. 5E and 5F are diagrams illustrating an operating method of thedriver assistance apparatus illustrated in FIGS. 5A to 5D.

First, FIG. 5E is a diagram showing an example of a situation in frontof the photographed by the stereo camera 200 b included in the vehicle.In particular, the vehicle front situation may be displayed as a bird'seye view.

As seen from the drawing, a first lane 642 a, a second lane 644 a, athird lane 646 a, and a fourth lane 648 a may be viewed to the rightfrom the left, a construction zone 610 a may be positioned between thefirst lane 642 a and the second lane 644 a, a first preceding vehicle620 a may be positioned between the second lane 644 a and the third lane646 a, and a second preceding vehicle 630 a may be positioned betweenthe third lane 646 a and the fourth lane 648 a.

Then, FIG. 5F shows an example in which a situation in front of thevehicle recognized by the driver assistance apparatus is displayed alongwith various information items. In particular, an image shown in FIG. 5Dmay be displayed on a display device 600 for the vehicle.

Unlike FIG. 5E, FIG. 5F shows an example in which information isdisplayed based on an image photographed by the stereo camera 200.

As seen from the drawing, a first lane 642 b, a second lane 644 b, athird lane 646 b, and a fourth lane 648 b may be viewed to the rightfrom the left, a construction zone 610 b may be positioned between thefirst lane 642 b and the second lane 644 b, a first preceding vehicle620 b may be positioned between the second lane 644 b and the third lane646 b, and a second preceding vehicle 630 b may be positioned betweenthe third lane 646 b and the fourth lane 648 b.

The camera 200 may perform signal processing on stereo imagesphotographed by the stereo cameras 200 b to verify objects correspondingto the construction zone 610 b, the first preceding vehicle 620 b, andthe second preceding vehicle 630 b. In addition, the first lane 642 b,the second lane 644 b, the third lane 646 b, and the fourth 648 b may beverified.

Meanwhile, the drawing illustrates an example in which edges arehighlighted to indicate object verification of the construction zone 610b, the first preceding vehicle 620 b, and the second preceding vehicle630 b.

Meanwhile, the camera 200 may calculate distance information on theconstruction zone 610 b, the first preceding vehicle 620 b, and thesecond preceding vehicle 630 b based on the stereo images photographedby the stereo camera 200 b.

The drawing illustrates an example in which first distance information611 b, second distance information 621 b, and third distance information631 b, which are calculated to correspond to the construction zone 610b, the first preceding vehicle 620 b, and the second preceding vehicle630 b, respectively, are displayed.

The camera 200 may receive sensor information of the vehicle from thecontroller 170 or the internal sensing unit 125. In particular, thecamera 200 may receive vehicle speed information, gear information, yawrate information indicating speed at which a rotation angle (yaw angle)of the vehicle is changed, and angle information of the vehicle may bereceived, and these information items may be displayed.

The drawing illustrates an example in which vehicle speed information672, gear information 671, and yaw rate information 673 are displayed onan upper portion 670 of a front field-of-view image of the vehicle andvehicle angle information 682 is displayed on a lower portion 680 of thefront field-of-view image of the vehicle or, alternatively, there may bevarious examples. In addition, vehicle width information 683 and roadcurvature information 681 may be displayed along with the vehicle angleinformation 682.

FIG. 6 is a flowchart illustrating operation of a driver assistanceapparatus according to an embodiment of the present invention.

Referring to FIG. 6, the processor 470 may acquire a vehicle externalimage (S610). The processor 470 may acquire a front field-of-view imageof the vehicle photographed by the camera 200. The processor 470 maydetect at least one traffic sign based on the front field-of-view imageof the vehicle (S620).

Here, the traffic sign may include at least one of a traffic signalmarked on a road surface, a traffic signal marked on a trafficsignboard, a traffic lamp signal, a construction zone warning signal, atraffic signal attached to another vehicle, or a hand signal.

The processor 470 may acquire travel information of the vehicle (S630).

Here, the travel information of the vehicle may include at least one oftravel road information, travel lane information, destinationinformation, route information, turn signal information, steeringinformation, heading information, or vehicle model information.

The processor 470 may determine whether the traffic sign fits the travelinformation (S640).

When the traffic sign fits the travel information, the processor 470 mayperform control to output the detected traffic sign (S650).

Meanwhile, when a plurality of traffic signs is detected, the processor470 may perform control to output a traffic sign having a higherpriority level among the plurality of traffic signs according to apriority order which is stored in the memory 440 with respect to theplurality of traffic signs.

Meanwhile, when a plurality of items of travel information is acquired,the processor 470 may perform control to output a traffic sign fittingan item of travel information having a higher priority level among theplurality of items of travel information according to a priority orderwhich is stored in the memory 440 with respect to the plurality oftravel information.

When the traffic sign does not fit the travel information, the processor470 may perform control not to output the detected traffic sign (S660).

FIG. 7 is a diagram illustrating an operation of detecting a trafficsign according to an embodiment of the present invention. Referring toFIG. 7, the processor 470 may receive an image photographed by thecamera 200. Reference numeral 700 indicates an example of an imagecorresponding to a predetermined frame in the image photographed by thecamera 200.

Based on the received image, the processor 470 may detect and acquiretraffic signs 711, 712, 713, 714, 715, and 716.

On a road surface, traffic signs may be marked, such as various roadsurface markings (e.g., a center lane, a U-turn line, a bus lane, and alane change not-allowed line) or direction arrows (e.g., a left turn,straight running, a right turn, and a U-turn).

The processor 470 may detect the traffic sign 712 marked on a roadsurface from the received image.

A traffic sign, such as a caution sign, a regulatory sign, aninstruction sign, or an auxiliary sign, may be marked on the trafficsignboard.

The processor 470 may detect the traffic sign 711 marked on a trafficsignboard from the received image.

The processor 470 may acquire the traffic sign 713 by detecting a coloror position of light output from a traffic lamp based on a vehicleexternal image.

The processor 470 may acquire the traffic sign 715 by detecting at leastone of a traffic corn, a temporary traffic signboard, a traffic baton,or a guide light device based on a vehicle external image.

The processor 470 may acquire the traffic sign 716 by detecting anothervehicle based on a vehicle external image and detecting a traffic signattached to a body, a bumper, or a glass of another vehicle.

The processor 470 may acquire the traffic sign 714 by detecting a handsignal by a traffic police officer based on a vehicle external image.For example, the processor 470 may detect characteristics of trafficpolice officer cap and uniform and compare the detected characteristicswith data pre-stored in the memory 440, thereby detecting a trafficpolice officer. When the traffic police officer is detected, theprocessor 470 may detect a predetermined motion of the traffic policeofficer and compare the detected motion with data pre-stored in thememory 440, thereby detecting a hand signal.

FIG. 8 is a diagram illustrating an operation of a driver assistanceapparatus in response to detection of a plurality of traffic signsaccording to an embodiment of the present invention.

Referring to FIG. 8, the processor 470 may detect a plurality of trafficsigns 811, 812, and 813 from a received image.

The memory 440 may store a priority order of the plurality of trafficsigns.

When the plurality of detected traffic signs fits travel information,the processor 470 may perform control to output a traffic sign having ahigher priority level among the plurality of traffic signs according tothe priority order stored in the memory 440. Here, the plurality oftraffic signs may include items of information contradictory to eachother.

For example, the processor 470 may detect the traffic sign 812 based ona traffic lamp signal, and the traffic sign 813 based on a trafficsignboard. In this case, the processor 470 may perform control to outputthe traffic sign 812 based on the traffic lamp according to the priorityorder stored in the memory 440. In this case, the processor 470 mayperform control not to output the traffic sign 813 based on the trafficsignboard.

For example, the processor 470 may detect both the traffic sign 812based on a traffic lamp, and the traffic signboard 811 on which a stopsignal is marked. In this case, the processor 470 may perform control tooutput the stop signal marked on the traffic signboard 811 according tothe priority order stored in the memory 440.

For example, the processor 470 may detect a hand signal 814 as well asthe traffic sign 812 based on a traffic lamp. In this case, theprocessor 470 may perform control to output the hand signal 814according to the priority order stored in the memory 440.

FIG. 9 is a diagram illustrating an operation of outputting a trafficsign according to an embodiment of the present invention.

Referring to FIG. 9, the processor 470 may control the display unit 451to display traffic signs 931 and 932. Specifically, the processor 470may control the display unit 451 to display graphic images 931 and 932corresponding to traffic signs.

For example, the processor 470 may control the display unit 451 todisplay a text 931 corresponding to a detected traffic sign.

For example, the processor 470 may control the display unit 451 todisplay an image 932 corresponding to a detected traffic sign. Here, theimage 932 may be an image of a traffic sign region in an image receivedby the processor 470 from the camera 200.

Meanwhile, the processor 470 may output a traffic sign through the soundoutput unit 452.

For example, the processor 470 may output a voice of contentcorresponding to a traffic sign.

FIG. 10 is a diagram illustrating a driver assistance apparatus inresponse to acquisition of a plurality of items of travel informationaccording to an embodiment of the present invention.

Referring to FIG. 10, the processor 470 may receive a plurality of itemsof travel information. The plurality of items of travel information maybe items of information contradictory to each other.

As illustrated in FIG. 10, before entering an intersection, theprocessor 470 may receive destination information and right-turn signalinformation. The destination information indicates a straight forwarddirection at the intersection. While the vehicle 100 is traveling in therightmost lane, a right-turn signal input may be received.

Meanwhile, the processor 470 may detect a traffic sign of a firsttraffic lamp 1011 indicative of straight running, and a traffic sign ofa second traffic lamp 1012 indicative of a right turn.

Meanwhile, turn signal information may be stored in the memory 440 witha priority level higher than that of the destination information.

The processor 470 may determine whether the first traffic lamp 1011 andthe second traffic lamp 1012 fit the turn signal information. The firsttraffic lamp 1011 is a traffic lamp for straight running and thus doesnot fit the turn signal information. The second traffic lamp 1012 is atraffic lamp for a right turn and thus fits the turn signal information.

The processor 470 may control the output unit 450 so as to output thetraffic sign of the second traffic lamp 1012 fitting the turn signalinformation.

By performing control in the above manner, the driver assistanceapparatus 400 may output a traffic sign fitting a driver's intention andmy not output a traffic sign not fitting the driver's intention.

FIG. 11 is a diagram illustrating an operation of outputting a trafficsign based on distance to a plurality of traffic signs according to anembodiment of the present invention.

Referring to FIG. 11, the processor 470 may detect a plurality oftraffic signs 1111 and 1112 based on a received image. Information itemsincluded in each of the plurality of traffic signs may be contradictoryto each other.

The processor 470 may calculate a distance between the vehicle 100 andeach of the plurality of traffic signs 1111 and 1112.

When the plurality of detected traffic signs fits travel information ofthe vehicle, the processor 470 may control the output unit 450 to outputthe traffic sign 1111 closest to the vehicle 100 among the plurality oftraffic signs.

Information indicating that the traffic sign 1111 closest to the vehicle100 needs to apply to the vehicle at a time of traveling is included. Inthis case, there is an advantage in that information most appropriatefor the vehicle 100 is provided to a driver.

FIG. 12 is a diagram illustrating an operation of outputting a trafficsign in response to detection of a plurality of traffic signs accordingto an embodiment of the present invention.

Referring to FIG. 12, the processor 470 may detect both a first trafficsign 1211 and a second traffic sign 1212 from a received image.

The first traffic sign may be a traffic sign based on a traffic lamp forstraight running. The second traffic sign may be a traffic signboardindicative of a permitted left-turn.

The processor 470 may receive route information of the vehicle 100. Whenthe vehicle 100 needs to travel straight ahead according to the routeinformation, the processor 470 may control the output unit 450 to outputthe first traffic sign. In addition, the processor 470 may performcontrol not to output the second traffic sign which does not fit theroute information.

FIG. 13 is a diagram illustrating an operation of outputting an alarm bya driver assistance apparatus according to an embodiment of the presentinvention.

Referring to FIG. 13, the processor 470 may receive state information ofthe vehicle based on a driving manipulation signal through the interfaceunit 430.

The processor 470 may determine whether the state information fitsoutput traffic sign information.

If the state information does not fit the output traffic signinformation, the processor 470 may output an alarm through the outputunit 450.

As illustrated in FIG. 13, the processor 470 may detect a speed limittraffic sign marked on a traffic signboard from a received image. Whenthe traffic sign fits travel information of the vehicle, the processor470 may perform control to output the traffic sign.

Then, the processor 470 may receive vehicle speed information based on adriving manipulation signal. If a speed of the vehicle exceeds speedlimit corresponding to the traffic sign, the processor 470 may output analarm through the output unit 450.

FIG. 14 is a diagram illustrating an operation of a driver assistanceapparatus in response to entrance to a rampway according to anembodiment of the present invention.

Referring to FIG. 14, the processor 470 may acquire rampway enteringsituation information.

The processor 470 may acquire rampway entering situation information byreceiving navigation information. In another example, the processor 470may acquire rampway entering situation information based on a road shapedetected from a vehicle external image. In yet another example, theprocessor 470 may acquire rampway entering situation information bydetecting a traffic signboard indicative of a rampway.

The processor 470 may perform control to output a traffic sign whichfits travel information among a plurality of traffic signs placed on aplurality of paths forming the rampway.

As illustrated in FIG. 14, the processor 470 may detect a first trafficsign 1411, a second traffic sign 1412, and a third traffic sign 1413.

The first traffic sign 1411 corresponds to a first path among pathsforming the rampway. That is, the first sign 1411 is a signal providedto a vehicle traveling on the first path.

The second traffic sign 1412 and the third traffic sign 1413 correspondto a second path among the paths forming the rampway. That is, thesecond traffic sign 1412 and the third traffic sign 1413 are signalsprovided to a vehicle which is traveling on the second path.

The processor 470 may receive travel information of the vehicle. Thetravel information of the vehicle may include at least one of travelroad information, travel lane information, destination information,route information, turn signal information, steering information,heading information, or vehicle model information.

The processor 470 may select one of the first path and the second pathbased on the travel information of the vehicle. For example, theprocessor 470 may select one of the first path and the second path basedon at least one of travel lane travel information, destinationinformation, route information, turn signal information, steeringinformation, or heading information.

If the first path is selected, the processor 470 may control the outputunit 450 to output the first traffic sign 1411 corresponding to thefirst path.

FIG. 15 is a diagram illustrating an operation of a driver assistanceapparatus in response to detection of a plurality of traffic lampsaccording to an embodiment of the present invention.

Referring to FIG. 15, the processor 470 may detect a first traffic lamp1511 and a second traffic lamp 1512 based on a received image.

The processor 470 may calculate a distance between the vehicle 100 andthe first traffic lamp 1511. The processor 470 may calculate a distancebetween the vehicle 100 and the second traffic lamp 1512. As illustratedin FIG. 15, the first traffic lamp 1511 may be closer to the vehicle 100than the second traffic lamp 1512 is.

The processor 470 may determine whether the first traffic lamp 1511 andthe second traffic lamp 1512 fit travel information of the vehicle.

If the first traffic lamp 1511 and the second traffic lamp 1512 fit thetravel information of the vehicle, the processor 470 may perform controlto output a traffic sign corresponding to the first traffic lamp 1511.

In the case where a traffic sign of the first traffic lamp 1511indicates Go and a traffic sign of the second traffic lamp 1512indicates Stop, when both the traffic sign of the first traffic lamp1511 and the traffic sign of the second traffic sign 1512 are output, adriver may feel confused. In this case, only the traffic sign of thefirst traffic lamp 1511 closer to the vehicle 10 may be output,preventing the confusion to the driver.

FIG. 16 is a diagram illustrating an operation of outputting a trafficsign fitting line information by a driver assistance apparatus accordingto an embodiment of the present invention.

Referring to FIG. 16, the vehicle 100 may travel in a lane among aplurality of lanes 1621, 1622, and 1623 on a road. In proximity to anintersection, the plurality of lines may be defined as a left-turn lane1621, a straight running lane 1622, a right-turn lane 1623,respectively.

The processor 470 may acquire lane information on a lane in which thevehicle 100 is traveling among the plurality of lanes 1621, 1622, and1623. For example, the processor 470 may acquire the lane information ona lane in which the vehicle 100 is traveling, by receiving navigationinformation. For example, the processor 470 may acquire the laneinformation on a lane in which the vehicle 100 is traveling, bydetecting the lane from a received image.

Meanwhile, the processor 470 may detect a plurality of traffic signs1611, 1612, and 1613 based on a front field-of-view image of thevehicle.

The processor 470 may perform control to output a traffic sign fittingto lane information among the plurality of traffic signs 1611, 1612, and1613.

For example, a plurality of traffic sigs may include a first trafficsign 1611 based on a traffic signboard for a permitted left turn of thevehicle 100, and a second traffic sign 1612 based on a traffic lamp forstraight running of the vehicle 100.

If the vehicle 100 is traveling in the left-turn lane 1621, theprocessor 470 may acquire situation information indicating a situationthat the vehicle 100 is traveling in the left-turn lane 1621.

The processor 470 may determine whether the first traffic sign 1611 andthe second traffic sign 1612 fit the situation information indicating asituation that the vehicle 100 is traveling in the left-turn lane 1621.In this embodiment, the first traffic sign 1611 fits the situationinformation, and the second traffic sign 1612 does not fit the situationinformation.

The processor 470 may perform control such that the first traffic isoutput and the second traffic is not output.

In another example, a plurality of traffic signs may include a firsttraffic sign 1613 based on a first traffic lamp for a right turn of thevehicle 100, and a second traffic sign 1612 based on a second trafficlamp for straight running of the vehicle 100.

If the vehicle 100 is traveling in the right-turn lane 1623, theprocessor 470 may acquire situation information indicating a situationthat the vehicle 100 is traveling in the right-turn lane 1623.

The processor 470 may determine whether the first traffic sign 1613 andthe second traffic sign 1612 fit the situation information indicating asituation that the vehicle 100 is traveling in the right-turn lane. Inthis situation, the first traffic sign 1613 fits the situationinformation, and the second traffic sign 1612 does not fit the situationinformation.

The processor 470 may perform control such that the first traffic signis output and the second traffic sign is not output.

FIG. 17 is a diagram illustrating an operation of a driver assistanceapparatus in a congested area according to an embodiment of the presentinvention.

Referring to FIG. 17, the processor 470 may acquire vehicle speedinformation. When a speed of the vehicle is equal to or lower than areference speed, the processor 470 may perform control not to output atraffic sign 1710 even in the case where the traffic sign 1710 fitstravel information.

In this case, the traffic sign 1710 may be far from the vehicle 100 at areference distance or more.

When a speed of the vehicle is equal to or lower than the referencespeed due to traffics, the traffic sign 1710 at the reference distanceor more from the vehicle 100 is not appropriate for the congestionsituation. In this case, the traffic sign 1710 not appropriate for thecongestion situation, thereby preventing confusion to a driver.

FIG. 18 is a diagram illustrating an operation of a driver assistanceapparatus depending on a vehicle model according to an embodiment of thepresent invention.

Referring to FIG. 18, a traffic sign necessary to provide may differdepending a type of the vehicle 100 even in the case of traveling on thesame road.

For example, in the case of traveling on a highway, speed limit maydiffer depending on whether a vehicle travelling on the highway is apassenger vehicle or a truck.

As illustrated in FIG. 18, speed limit for passenger vehicles may be 100km/h, and speed limit for trucks may be 60 km/h. In this case, a firsttraffic signboard 1811 for passenger vehicles and a second trafficsignboard 1812 for trucks may be arranged together on the highway.

The processor 470 may detect a first traffic sign based on the firsttraffic signboard 1811, and a second traffic sign based on the secondtraffic signboard 1812.

The processor 470 may determine whether the first traffic sign and thesecond traffic sign fits vehicle model information which is included intravel information of the vehicle. The vehicle model information may bestored in the memory 130 of the vehicle 100.

If the vehicle 100 is a passenger vehicle, the first traffic sign fitsthe vehicle model information and the second traffic sign does not fitthe vehicle model information.

The processor 470 may perform control such that the first traffic signfitting the vehicle model information is output and the second trafficsign not fitting the vehicle model information is not output.

The above-described methods can be implemented in a program recordedmedium as computer-readable codes. The computer-readable media mayinclude all kinds of recording devices in which data readable by acomputer system are stored. The computer-readable media may include HardDisk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM,RAM, CD-ROM, magnetic tapes, floppy discs, optical data storage devices,and the like, and also include carrier-wave type implementations (e.g.,transmission via Internet). Further, the computer may include aprocessor or a controller. Accordingly, the above-mentioned detaileddescription should be considered only for illustrative purposes, not forrestrictive purposes. The scope of the present invention should bedetermined by rational interpretation of the claims, and allmodifications within equivalent ranges of the present invention shouldbe construed as being included in the scope of the present invention.

What is claimed is:
 1. A driver assistance apparatus for a vehicle,comprising: a camera configured to acquire a front field-of-view imageof the vehicle; an output unit; and a processor configured to detect atleast one traffic sign based on the front field-of-view image of thevehicle, determine whether the traffic sign fits travel information ofthe vehicle, and, when the traffic sign fits the travel information,perform control to output the traffic sign through the output unit,wherein the processor is further configured to: acquire vehicle speedinformation, and when a speed of the vehicle is equal to or lower than areference speed, perform control not to output the traffic sign even ina case where the traffic sign fits the travel information.
 2. The driverassistance apparatus of claim 1, wherein the processor is furtherconfigured to, when the traffic sign does not fit the travelinformation, perform control not to output the traffic sign.
 3. Thedriver assistance apparatus of claim 1, further comprising an interfaceunit, wherein the processor is further configured to, when the trafficsign fits the travel information, provide at least one of a steeringcontrol signal, an acceleration control signal, or a brake controlsignal through the interface unit based on the traffic sign.
 4. Thedriver assistance apparatus of claim 1, wherein the traffic signcomprises at least one of a traffic signal marked on a road surface, atraffic signal marked on a traffic signboard, a traffic lamp signal, aconstruction zone warning signal, a traffic signal attached to anothervehicle, or a hand signal.
 5. The driver assistance apparatus of claim4, further comprising a memory configured to store a priority order of aplurality of traffic signs, wherein the processor is further configuredto, when the plurality of traffic signs fits the travel information,perform control based on the priority order to output a traffic signhaving a higher priority level among the plurality of traffic signs. 6.The driver assistance apparatus of claim 5, wherein the memory isfurther configured to store the priority order in which the traffic lampsignal has a priority level higher than a priority level of the trafficsignal marked on the road surface or the traffic signal marked on thetraffic signboard.
 7. The driver assistance apparatus of claim 5,wherein the memory is further configured to store the priority order inwhich the hand signal has a priority level higher than a priority levelof the traffic lamp signal.
 8. The driver assistance apparatus of claim5, wherein the memory is further configured to store the priority orderin which a stop signal marked on the traffic signboard has a highestpriority level.
 9. The driver assistance apparatus of claim 1, whereinthe travel information comprises at least one of travel roadinformation, travel lane information, destination information, routeinformation, turn signal information, steering information, headinginformation, or vehicle model information.
 10. The driver assistanceapparatus of claim 9, further comprising: a memory configured to store apriority order of a plurality of items of travel information, whereinthe processor is further configured to, when the plurality of items oftravel information, determine whether the traffic sign fits an item oftravel information having a higher priority level among the plurality ofitems of travel information, based on the priority order.
 11. The driverassistance apparatus of claim 10, wherein the memory is furtherconfigured to store the priority order in which the turn signalinformation has a priority level higher than a priority level of thedestination information, and wherein the processor is further configuredto, when the turn signal information and the destination information donot fit each other, determine whether the traffic sign fits the turnsignal information.
 12. The driver assistance apparatus of claim 1,wherein the processor is further configured to: detect a plurality oftraffic signs based on the front field-of-view image of the vehicle;calculate a distance between a vehicle and each of the plurality oftraffic signs; and perform control to output a traffic sign closest tothe vehicle among the plurality of traffic signs which fits the travelinformation.
 13. The driver assistance apparatus of claim 1, wherein theprocessor is further configured to: detect a first traffic sign and asecond traffic sign based on the front field-of-view image of thevehicle; and when the first traffic sign fits the travel information andthe second traffic sign does not fit the travel information, performcontrol to output the first traffic sign through the output unit and notto output the second traffic sign.
 14. The driver assistance apparatusof claim 1, further comprising an interface unit, wherein the processoris further configured to, when state information of the vehicle receivedbased on a driving manipulation signal through the interface unit doesnot fit output traffic sign information, output an alarm through theoutput unit.
 15. The driver assistance apparatus of claim 1, wherein theprocessor is further configured to: acquire rampway entering situationinformation related to a rampway; and perform control to output atraffic sign which fits the travel information among a plurality oftraffic signs placed on a plurality of paths forming the rampway. 16.The driver assistance apparatus of claim 1, wherein the processor isfurther configured to: detect a plurality of traffic lamps based on thefront field-of-view image of the vehicle; and perform control to outputa traffic sign generated by a traffic lamp closest to the vehicle amonga plurality of traffic lamps fitting the travel information.
 17. Thedriver assistance apparatus of claim 1, wherein the processor is furtherconfigured to: acquire lane information on a lane in which a vehicle istravelling; detect a plurality of traffic signs based on the frontfield-of-view image of the vehicle; and perform control to output atraffic sign which fits the lane information among the plurality oftraffic signs.
 18. A driver assistance apparatus for a vehicle,comprising: a camera configured to acquire a front field-of-view imageof the vehicle; an output unit; and a processor configured to detect atleast one traffic sign based on the front field-of-view image of thevehicle, determine whether the traffic sign fits travel information ofthe vehicle, and, when the traffic sign fits the travel information,perform control to output the traffic sign through the output unit,wherein the processor is further configured to: acquire lane informationon a lane in which a vehicle is travelling, detect a plurality oftraffic signs based on the front field-of-view image of the vehicle, andperform control to output a traffic sign which fits the lane informationamong the plurality of traffic signs, wherein the plurality of trafficsigns comprises a first traffic sign based on a traffic signboard for apermitted left turn of the vehicle, and a second traffic sign based on atraffic lamp for straight running of the vehicle, and wherein theprocessor is further configured to: acquire situation informationindicating a situation that the vehicle is travelling on a left-turnlane, and perform control to output the first traffic sign and not tooutput the second traffic sign.
 19. A driver assistance apparatus for avehicle, comprising: a camera configured to acquire a frontfield-of-view image of the vehicle; an output unit; and a processorconfigured to detect at least one traffic sign based on the frontfield-of-view image of the vehicle, determine whether the traffic signfits travel information of the vehicle, and, when the traffic sign fitsthe travel information, perform control to output the traffic signthrough the output unit, wherein the processor is further configured to:acquire lane information on a lane in which a vehicle is travelling,detect a plurality of traffic signs based on the front field-of-viewimage of the vehicle, and perform control to output a traffic sign whichfits the lane information among the plurality of traffic signs, whereinthe plurality of traffic signs comprises a first traffic sign based on afirst traffic lamp for a right turn of the vehicle, and a second trafficsign based on a second signal lamp for straight running of the vehicle,and wherein the processor is further configured to: acquire situationinformation indicating a situation that the vehicle is travelling in aright-turn lane, and perform control to output the first traffic signand not to output the second traffic sign.